Skip to content

Add import rate and timestamp ordering guidance to migrations docs - #19788

Open
darkopia wants to merge 2 commits into
masterfrom
posthog/docs-migrate-rate-limit-ordering
Open

Add import rate and timestamp ordering guidance to migrations docs#19788
darkopia wants to merge 2 commits into
masterfrom
posthog/docs-migrate-rate-limit-ordering

Conversation

@darkopia

@darkopia darkopia commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Changes

The historical migrations overview did not say how fast you can import, or in what order. Customers who write their own import scripts have no guidance, and unthrottled backfills with scrambled timestamps destabilize ingestion.

This adds two subsections to Best practices for migrations:

  • Limit your import rate — keep total throughput below a few thousand events per second across all workers, with a pointer to managed migrations, which handle this for you.
  • Send events in timestamp order — batch events by a short time range (one day is a good default) and import oldest first. Explains why: mixed timestamps make ClickHouse write many small parts.

Content-only change to contents/docs/migrate/index.mdx. No navigation, component, or visual changes, so no screenshots and no redirect needed.

Checklist

  • I've read the docs and/or content style guides.
  • Words are spelled using American English
  • Use relative URLs for internal links
  • I've checked the pages added or changed in the Vercel preview build
  • If I moved a page, I added a redirect in vercel.json (n/a — no pages moved)

Created with PostHog from a Slack thread

Adds two best practices to the historical migrations overview: keep DIY import throughput below a few thousand events per second, and batch events by time range (roughly one day) and import oldest first.

Generated-By: PostHog Desktop
Task-Id: 01a3e0db-9f18-40a1-b97c-cd050581624f
@github-actions github-actions Bot added the content PR only touches files under contents/ label Aug 28, 2026
@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Deploy preview

Status Details Updated (UTC)
🟢 Ready View preview Aug 28, 2026 01:18AM

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Vale prose linter → found 0 errors, 1 warnings, 0 suggestions in your markdown

Full report → Copy the linter results into an LLM to batch-fix issues.

Linter being weird? Update the rules!

contents/docs/migrate/index.mdx — 0 errors, 1 warnings, 0 suggestions
Line Severity Message Rule
55:9 warning 'resumability' is a possible misspelling. PostHogBase.Spelling

Generated-By: PostHog Desktop
Task-Id: 01a3e0db-9f18-40a1-b97c-cd050581624f
@darkopia
darkopia marked this pull request as ready for review August 28, 2026 01:05
@github-actions

Copy link
Copy Markdown
Contributor

Bundle report

Total JS (gzip)

8.15 MiB (no change)

Eager graph (modules shipped in each entrypoint's initial chunks)

Entrypoint Eager size Budget Modules
app 16.90 MiB (no change) report-only 2035
Largest modules in the app closure
Module Size
./src/data/mcp-tools.json 1053.0 KiB
css ./node_modules/.pnpm/css-loader@5.2.7_webpack@5.101.3/node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[1]!./node_modules/.pnpm/postcss-loader@4.3.0_postcss@8.5.6_webpack@5.101.3/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[2]!./src/styles/global.css 754.4 KiB
./src/components/Stickers/Stickers.tsx 696.4 KiB
./node_modules/.pnpm/@radix-ui+react-icons@1.3.2_react@18.3.1/node_modules/@radix-ui/react-icons/dist/react-icons.esm.js 481.4 KiB
./node_modules/.pnpm/rehype-raw@7.0.0/node_modules/rehype-raw/lib/index.js + 29 modules 395.1 KiB
./src/hooks/useCustomers.tsx + 55 modules 370.0 KiB
./node_modules/.pnpm/@posthog+icons@0.36.6_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js 354.8 KiB
./node_modules/.pnpm/react-markdown@8.0.7_@types+react@16.14.66_react@18.3.1/node_modules/react-markdown/lib/react-markdown.js + 88 modules 351.4 KiB
./src/components/ProductComparisonTable/index.tsx + 126 modules 301.7 KiB
./node_modules/.pnpm/cloudinary-core@2.14.0_lodash@4.17.21/node_modules/cloudinary-core/cloudinary-core.js 281.9 KiB
./src/components/SearchUI/index.tsx + 87 modules 273.0 KiB
./node_modules/.pnpm/@posthog+brand@0.8.0_react@18.3.1/node_modules/@posthog/brand/dist/generated/hoggies/svg/magnifying-glass.mjs 254.7 KiB
./node_modules/.pnpm/framer-motion@10.18.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/framer-motion/dist/es/render/dom/motion.mjs + 109 modules 253.9 KiB
./node_modules/.pnpm/d3@7.9.0/node_modules/d3/src/index.js + 208 modules 247.4 KiB
./src/components/Pricing/PricingSlider/Slider.tsx + 87 modules 240.1 KiB

Eager-graph budgets are report-only until a baseline is established. Sizes are gzip of public/**/*.js; eager size is webpack module source bytes for the modules actually shipped in the entrypoint's initial chunks (post-tree-shake).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

content PR only touches files under contents/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant